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ABSTRACT 
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Improvement Amendments to Chapter 1 (now Title I) have enabled broad 
expansion of Title I schoolwide projects. The regulatory changes provide 
funding to entire schools, rather than targeting services to meet the needs 
of the most disadvantaged subpopulations. This paper reviews what is known 
about Title I schoolwide projects and their effectiveness, focusing on three 
aspects: characteristics of schools and districts with schoolwide projects; 
programmatic and organizational characteristics of schoolwide project schools 
and districts; and student-performance outcomes. The study reviewed 12 
evaluative publications, all of which were based on data collected between 
1985 and 1995. The paper presents some cautions about data interpretation and 
suggests implications for future evaluation of schoolwide projects in schools 
and districts. It concludes that evaluations of student performance have 
yielded mixed and inconsistent results. Reliable, longitudinal evaluations 
must continue to be conducted beyond the initial phase. Three tables are 
included. Appendices contain district-level studies of program effectiveness 
and student performance. (Contains 23 references.) (LMI) 
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Abstract 



The Hawkins-Stafford Elementary and Secondary School Improvement Amendments to 
Chapter 1 (now Title I), passed in 1988, have enabled broad expansion of Title I schoolwide 
projects to over 8,000 schoolwide projects during the 1995-96 school year. These regulatory 
changes have the potential to reduce the historically fragmented or "categorical” character of Title I 
programs and improve the effectiveness of entire schools rather than targeting services to meet the 
needs of the most disadvantaged subpopulations. Despite the dramatic increase in the number of 
schoolwide projects, however, there remain a number of questions about their effectiveness 
relative to traditional Title I programming. This paper presents a review of what is known about 
Title I schoolwide projects and their effectiveness, focusing on three aspects: characteristics of 
schools and districts with schoolwide projects; programmatic and organizational characteristics of 
schoolwide project schools and districts; and evidence of the effectiveness of schoolwide project 
schools, particularly in terms of student performance. In addition to reviewing these evaluation 
findings, we present a handful of cautions related to their interpretation. Finally, we suggest 
implications for future evaluation and for administrators in schools and districts with schoolwide 
projects. 
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Background 

Since the Great Society era of the 1960s, the federal grants-in-aid system has been used as the 
primary tool to address equity issues in elementary and secondary education. One of the major 
federal categorical programs that survived partisan shifts is compensatory education, enacted as 
Title I of the Elementary and Secondary Education Act in 1 965. Consistent with the redistributive 
purpose in the original act, Title I has provided supplementary resources to schools with large 
numbers of low-income students throughout the past three decades. For Fiscal Year 1995, the 
Clinton Administration allocated almost $8 billion in Title I funds to over 4 million students. 
Enjoying bipartisan support over the years, Title I was not targeted for elimination by the 
Republican majority in the Congress during 1995 and 1996. 

Federal funding support notwithstanding, policymakers in Washington have paid increasing 
attention to classroom learning among disadvantaged students in recent years. Since the 
publication of A Nation at Risk in 1983, there have been renewed concerns for blending Title I 
with a core academic curriculum (Kirst, 1988). Policy analysts have directed their attention to 
program redesign at the school level in ways that would strengthen the schools' overall 
organizational capacity to develop more comprehensive (i.e., less fragmentary) strategies for 
helping disadvantaged children (Wong & Wang, 1994; Millsap, et al., 1992; Commission on 
Chapter 1 , 1 992). Further, previous research has documented the stigmatizing impact of pullout 
programs on students, their disruptive impact on classroom instruction, and the organizational 
complexity resulting from the targeting provision that only means-tested, eligible children may be 
served (Glass & Smith, 1977; Leinhardt, et al, 1986; Winfield & Hawkins, 1993). 

In light of these concerns, the U.S. Congress passed the Hawkins-Stafford Elementary and 
Secondary School Improvement Amendments to Chapter 1 (now Title I) in 1988 which 
encouraged schoolwide projects in schools where at least 75% of the students come from low- 
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income backgrounds to operate them (without asking for local matching funds). The 1994 
Improving America's School Act further encouraged the adoption of schoolwide projects by 
lowering the eligibility threshold for schoolwide projects to schools with 50% low-income students 
by 1996-97. In big-city districts, the lower threshold permits virtually all Title I schools to operate 
schoolwide projects. These regulatory changes have the potential to reduce the historically 
fragmented or "categorical" character of Title I programs and improve the effectiveness of entire 
schools rather than targeting services to meet the needs of the most disadvantaged subpopulations. 
The schoolwide project option allows schools to develop instructional programs which coordinate 
resources between Title I and regular instructional programs in the school as well as foster 
increased programmatic coordination between federal, state, and local staff in schools that serve 
large numbers of disadvantaged students. 

As a result of these federal legislative changes, the number of schoolwide projects grew from 
fewer than 1,200 to over 4,500 between 1991 and 1995, representing an increase from 10% to 
47% of all eligible Title I schools. During 1995-96, over 8,000 schoolwide projects were in 
operation out of a total 16,853 eligible school sites. Despite this rapid expansion, however, little is 
known about the nature of schoolwide projects and their effectiveness relative to traditional Title I 
programming. 



The Need for a Synthesis of Title I Evaluation Findings 

At the time of the passage of the Improving America's School Act, there were few empirical 
studies on the implementation of schoolwide projects and only a handful of studies have been 
conducted during the years following the 1988 legislation. Nationwide evaluations of the first 
years of the Hawkins-Stafford reform suggested mixed effects of schoolwide projects and offered 
little information with respect to their impact on student performance (U.S. Department of 
Education, 1992; 1993b). Further, several evaluations were conducted within particular school 
districts and their findings have not yet been integrated. Thus, there is a need to develop a more 
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systematic understanding of how schoolwide projects have worked during their initial years. As 
evaluation data continues to become available, it becomes increasingly important to integrate 
research findings and develop an informed picture of schoolwide project implementation thus far. 

This paper endeavors to synthesize what is known, to date, about the effectiveness of Title I 
schoolwide projects. In particular, our synthesis focuses on three primary aspects: 

• characteristics of schools and districts with schoolwide projects, 

• programmatic and organizational characteristics of schoolwide project schools and districts, 
and 

• evidence of the effectiveness of schoolwide project schools, particularly in terms of student 
performance. 

In addition to reviewing these research findings, we present a handful of cautions related to their 
interpretation. Finally, we suggest implications for future evaluation and for administrators in 
schools and districts with schoolwide projects. 

Overview of Evaluation Studies 

A total of twelve evaluative publications are reviewed in this analysis. [INCLUDE METHODS 
FOR IDENTIFYING/LOCATING STUDIES ?] Their characteristics are summarized in Tables 1 
and 2. All are based on data collected between 1985 and 1995 and were published in the early 
1990s. Two reports —the interim and final reports of the National Assessment of the Chapter 1 
Program— aggregate findings from a variety of related research studies (U.S. Department of 
Education, 1992; 1993b). The remaining ten present analyses of schoolwide projects at either the 
national or district level conducted by school districts, independent research organizations, and 
university researchers. The richest information about schoolwide project effectiveness in terms of 
student performance is from district-level studies of schoolwide project schools. The last six 
entries listed in Tables 1 and 2 describe these district-level studies. Some evaluation studies of 
Title I were excluded because reports to date present little information specifically related to 
schoolwide projects. Excluded, for example, are studies based on data from the Prospects 
database (U.S. Department of Education, 1993a) and those based upon state performance reports 
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(Sinclair & Gutmann, 1996). 

Samples and Data Collection Methods (see Table 1) 

Study designs range from large-scale surveys to in-depth case studies of particular school 
sites; ranging, for example, from a mail survey of nearly 2,000 schools to an in-depth case study 
of eleven schoolwide project schools. Thus, their findings are of varying generalizability. 
However, all focus primarily on elementary schools and, frequently, on a subset of grades 
between Kindergarten and eighth grade. Studies either base findings on samples of schoolwide 
project schools or present comparative analyses of schoolwide projects and schools with traditional 
Title I programming. Similarly, the samples of schools or districts used in these studies are either 
drawn randomly (to provide regional representativeness) or purposefully (to reflect variation across 
particular types of programs or student characteristics). 

Data collection methods include: administration of mail questionnaires and on-site 
interviewing with individuals at the school, district and state levels (e.g., principals, classroom 
teachers, instructional aides, state and district Title I coordinators, and parents); observation of 
classroom instruction and other school activities; use of extant data maintained by schools and 
districts; document review; and, administration of student achievement tests. The studies represent 
a fairly even mix of cross-sectional and longitudinal designs. In most cases, however, longitudinal 
studies focus exclusively on standardized measures of student achievement. 

Analytical Constructs and Data Analysis Methods (See Table 2) 

Table 2 summarizes the type of data collected and the data analysis methods for each study. 

The majority of studies present some basic descriptive information about: schoolwide project 
schools and districts (e.g., school poverty, school and district size, grades served) and the students 
and families served by school wide project schools (e.g., race/ethnicity, educational disadvantage, 
limited English proficiency). These data, in most cases, were collected via questionnaire or 
interview of individuals at the school, district, or state levels and through review of district data 
records. 
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Most studies devote their attention to the implementation process for schoolwide projects— in 
particular, elements of the proposal and planning process and the specific elements added as 
components of the schoolwide project. Components include both those related to school 
organization, management, and governance (e.g., hiring of additional staff, the establishment of 
shared decision making structures, and increased staff development or parent involvement 
activities) as well as those related to curriculum and instruction (e.g., the distinction of Title I 
services from those in the regular program, changes in classroom instruction, and the inclusion of 
components associated with effective schools). These data were collected via questionnaire and 
interview with individuals at the school, state, and district level as well as through observation at 
the school site. 

To assess schoolwide project effectiveness, studies used both qualitative assessments by 
principals or district staff (e.g., perceived advantages or disadvantages of schoolwide projects) and 
quantitative measures of student performance (i.e. test scores). The majority of studies that present 
student performance measures use normal curve equivalent (NCE) gains on norm-referenced tests. 
Nearly all studies which present norm-referenced test scores do so for both reading and math. 
Scores in all cases are from widely-used, standardized tests, such as the California Achievement 
Test (CAT), Metropolitan Achievement Test (MAT), and the Comprehensive Test of Basic S kill s 
(CTBS). Only one study considers an additional measure of student performance —student report 
card marks. Two studies also consider student attendance as an indicator of schoolwide project 
effectiveness. Finally, school participation in program improvement efforts is also used as an 
indicator of program effectiveness in three cases. 

A variety of analytic techniques were used to generate findings from these data. Many findings 
are based upon aggregation of results from questionnaires, interviews, and observation. These 
findings tend to be primarily descriptive (e.g., characteristics of schools and districts with 
schoolwide projects, or inventories of components of schoolwide projects). Analyses of student 
performance measures, in most cases, are longitudinal, comparing cohorts of students in schools 
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with and without schoolwide projects which have been matched on some other characteristics. 
Nearly all analyses present changes in NCE scores, averaged across grades within districts. In 
fewer cases, findings are based on performance measures from schoolwide project schools only, 
considering average, grade-level test scores or measures of attendance before and after the point of 
schoolwide project implementation. Finally, in one case, findings are presented that are based only 
on a single, point-in-time analysis of test scores from schools with and without schoolwide 
projects. 

Very few of the studies present analyses which link student performance measures to 
characteristics of school wide project schools or students. Two studies relate school characteristics 
(e.g., school-level organization, instructional strategies) to student achievement scores and three 
relate student characteristics (e.g., gender, race/ethnicity, prior achievement) to student 
achievement scores. 



Findings 

Characteristics of Schools and Districts with Schoolwide Projects 

A wealth of basic descriptive information about schools and districts with schoolwide projects 
originates from a survey of nearly 500 districts and nearly 2,000 schools with schoolwide projects 
conducted by Schenck and Beckstrom during the 1991-92 school year. Over half of the 
schoolwide projects operating in 1991-92 were in districts with 25,000 or more students. 
Schoolwide project schools reported an average poverty level of 85 percent and served an average 
of 70 percent educationally disadvantaged students. Over half of schoolwide project schools 
served students from urban areas and nearly a quarter served students from rural areas. The 
remaining quarter of schoolwide project schools served small towns/communities and suburban 
areas. 

Over three-fourths of schoolwide project schools were elementary schools; another 12.4% 
served elementary grades in combination with one or more secondary grades; and only 10% were 
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secondary schools. Schoolwide project schools tend to have higher percentages of African- 
American and Hispanic students than the districts in which they operate and nearly 20 percent of 
students enrolled in schoolwide project schools demonstrated limited English proficiency (Schenck 
& Beckstrom, 1993). 

Implementation Progress 

Nearly all studies devote some attention to the implementation process of schoolwide project 
schools, endeavoring to describe the particular changes at the school level that occurred as a result 
of the schools’ adoption of the schoolwide approach. Two general areas of implementation 
progress are presented: 1) elements of the proposal and planning process and 2) components of 
school organization, management, governance, and curriculum and instruction which have been 
initiated, strengthened, or modified as a result of schoolwide project implementation. 

Elements of the Proposal and Planning Process 

According to the 1991-92 survey of schools with schoolwide projects, “more flexibility in 
service delivery or instructional grouping” was cited most often (by 85 percent of principals) as a 
reason for establishing the schoolwide projects. “Better fit [of the Title I program] with the total 
school program”, “more discretion in the use of [Title I] funds”, and “access to additional funds” 
were also cited often —by 57 percent, 46 percent, and 21 percent of principals, respectively. Only 
1 1.7 percent of principals selected "a response to being identified for Title I program improvement" 
as a reason for seeking schoolwide project status. (Schenck & Beckstrom, 1993). 

Nonetheless, a great deal of evidence indicates that the endorsement of schoolwide projects by 
personnel at the district and state level exerts a strong influence. For example, a survey of Title I 
Coordinators during the 1990-91 school year indicates that, although only 28 percent of all districts 
with eligible schools had established schoolwide projects, 65 percent of districts with eligible 
schools established schoolwide projects in districts whose state education agencies have 
encouraged consideration of the option (Millsap, et al, 1992). Indeed, there are many examples of 
school districts and state education agencies initiating the planning and application process to 
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establish schoolwide projects (Philadelphia School District, 1992; Stringfield, et al, 1994; Wong, 
et al, 1996). Almost none of the schools indicated that the district played no role in applying for 
the schoolwide project; however, one-fifth indicated that state department staff played no role 
(Schenck & Beckstrom, 1993). 

Most schoolwide project schools spent relatively little time in the process of planning and needs 
assessment. Nearly half reported spending less than six months and almost 90 percent reported 
spending less than a year. Schools encouraged by the district or state to apply for schoolwide 
project status spent even less time planning. The relatively short planning period may be, in part, a 
result of district influence on the design of schoolwide projects, which was reported as a factor in 
three-quarters of schoolwide project schools. Most schools, however, involved administrators, 
teachers, parents, and other individuals in the planning process, although one study describes 
parents’ role in planning schoolwide projects as having been limited (Schenck & Beckstrom, 1993; 
Millsap, et al, 1992). 

Because a fundamental purpose of the schoolwide project option is to provide schools with 
greater flexibility to use resources to meet the needs of all students at the school, a central question 
is how schools have taken advantage of this flexibility. In particular, what types of school 
organization, management, and educational practices have been instituted, strengthened, or 
changed as a result of schoolwide project implementation. We focus on two general types of 
components of schoolwide projects: 1) components of schoolwide projects related to school 
organization, management, and governance and 2) components of schoolwide projects related to 
curriculum and instruction. 

Components Related to School Organization. Management, and Governance 

Schoolwide projects have allowed schools to introduce new activities or programs as well as 
strengthen existing ones. Evidence from the first few years of their operation shows that emphasis 
has been placed primarily on strengthening existing programs and activities (Schenck & 

Beckstrom, 1993; Stringfield, et al, 1994). Further, preliminary evidence indicates that the shift to 
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a schoolwide project is often accompanied by changes in school management such as a reduction 
of class size. Of those schools which introduced new activities or programs, the most frequently 
introduced were activities for "parent education/involvement" and "adoption/adaption of a generic 
instructional program or approach.” Many other programs and activities were strengthened, 
however. During their first years, schools introduced an average of one or two activities, 
programs or strategies and strengthened five or six (Schenck & Beckstrom, 1993). 

Reduced Class Size! Hiring of Additional Staff 

Reduced class size was identified as a component of schoolwide project implementation by 
principals in slightly more than half of schoolwide project schools. The average reduction in 
school class size was from 27 to 19 children. Despite the prevalence of this component, schools 
that reduced class size introduced and strengthened other activities to a similar extent as schools 
which did not reduce class size (Schenck & Beckstrom, 1993). Teachers have expressed 
enthusiasm about this component, arguing that reductions in class size allow them to enhance their 
curricula and instruction. In fact, class size has been reduced selectively in some cases, only for 
those teachers who express a willingness to tie the reduction to a change in their instructional 
practice. Finally, addition of new staff has not been limited to teachers. Counselors, social 
workers, school-family coordinators, and schoolwide project coordinators have also been hired as 
components of schoolwide projects (Millsap, et al, 1993). 

Staff Development 

Case studies of schools with schoolwide approaches indicate that successful implementation of 
a schoolwide project is linked, in part, to the provision of significant resources for staff in-service 
training in particular instructional techniques and continuing opportunities for staff interaction 
(Stringfield, et al, 1994). Preliminary implementation evidence shows that staff development 
activities have been implemented or significantly strengthened in over three-fourths of schoolwide 
project schools. During the first years of implementation, a majority of districts report that staff 
development at schoolwide projects schools is more inclusive of teachers and involves more total 
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hours than regular Title I schools. Staff development activities in schoolwide project schools have 
included training in reading/language arts instruction, instruction for low achieving students, and 
mathematics instruction (Schenck & Beckstrom, 1993). According to principal reports, the 
average teacher at a schoolwide project school receives an average of 29 hours of staff 
development, which is 6 hours more than that received by typical teachers in Title I schools 
without school wide projects (Schenck & Beckstrom, 1993; Millsap, et al, 1993). 

Shared Decision Making and Teacher Input 

The schoolwide project option encourages increased teacher input into decisions affecting the 
school, emphasizing teacher input into decisions about assessments. Based on reports by 
principals in a major urban school district, teachers indicate having only a moderate level of input 
into decisions affecting the school. The majority of teachers have some level of input into 
decisions about assigning students and teachers to classrooms, hiring staff, and selecting materials 
or purchasing hardware. Teachers, however, had the greatest input in decisions about selecting 
materials or purchasing hardware and had the least input in decisions about teacher assignment and 
replacement (Winfield & Hawkins, 1993). 

Parental Involvement 

Acknowledged as a key component of all Title I efforts, parental involvement has also been a 
component of many schoolwide projects. Parent education and involvement is reported to have 
been introduced or significantly strengthened in over three-fourths of schoolwide project schools. 
The most common types of parent involvement activities mentioned were: informal contact 
between parents and teachers, parent volunteering in the school, and parent help with student 
school work in the home. School-based adult education and family literacy programs were 
mentioned less frequently (Schenck & Beckstrom, 1993). 

Principal reports in a large urban school district indicate that, despite a range of parent 
involvement activities, small percentages of parents were actually involved with these activities. 

For example, only five to ten percent of parents were reported as having been involved in 
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volunteering in classrooms and attending workshops. Higher proportions of parents were 
involved in other school activities (e.g., receiving information from teachers or home 
demonstrators, monitoring homework, and attending PTA meetings); however, this occurred in 
relatively few schools (Winfield & Hawkins, 1993). Case studies of Title I schools with particular 
programmatic “special strategies” indicate that, of several approaches, schoolwide projects are 
particularly well-positioned to foster parent involvement (Stringfield, et al, 1994). 

Components of Schoolwide Projects Related to Curriculum and Instruction 
Distinction of Title I Services from the Regular Program 

A central component of the schoolwide project option is the provision of Title I activities and 
services to all students in Title I schools. One indicator of a program’s inclusiveness is the extent 
to which Title 1 services cannot be distinguished from services offered for all children (e.g., one 
that lacks pullout programs which serve only a subset of students). Sixty percent of schoolwide 
project school principals report that their schools operate programs in which Title I services are 
indistinguishable from services for all children. Among those schools in which Title I services 
were distinguishable from the regular program, the most common distinction was the provision of 
additional services to educationally disadvantaged students who would have received Title 1 
services in a traditional, targeted program. Only 12 percent of schools reported using a pullout 
model (Schenck & Beckstrom, 1993). Similarly, principals in a major urban school district 
reported that at least one-fourth of schools had pre-existing Title I programs that were still 
operating in the context of the schoolwide project after the first three years of implementation 
(Winfield & Hawkins, 1993). 

Classroom Instruction 

Perhaps the most critical components of schoolwide projects are those which have the potential 
to directly influence what takes place in the classroom. Schoolwide project principals reported 
having introduced or significantly strengthened the following components related to curriculum and 
instruction: computer assisted instruction (over three-fourths); provision of a coordinated and 
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integrated curriculum and supplemental instruction (two-thirds); and provision of an extended 
school day (less than one-fourth) (Schenck & Beckstrom, 1993). Some schoolwide project 
schools have adopted particular programs or curricula, such as “Reading Recovery” or “Success 
for All,” as part of their schoolwide projects (Millsap, et al, 1992). In fact, the majority of 
“Success for All” schools are Title I schoolwide projects (Slavin, et al, 1995). 

It is difficult, however, to obtain a clear picture of the particular ways in which the schoolwide 
project option actually impacts classroom instruction. In-depth, case study analysis (Stringfield, et 
al, 1994) begins to inform this question, however. There is also evidence that schoolwide 
projects increase the capacity of schools and teachers to provide instructional services more 
flexibly, as particular student needs arise; whereas traditional Title I pullout programs have 
typically required a more formal process of student selection (Millsap, et al, 1992). 

Components Associated with Effective Schools 

Another important indicator of schoolwide project implementation progress is the inclusion of 
components associated with effective schools. State Title I coordinators reported that 62 percent of 
schoolwide projects in their states incorporated components of effective schools programs as a 
main feature of their programs (Turnbull et al, 1990; U.S. Department of Education, 1992). A 
survey of district coordinators indicates that a number of effective schools components were 
implemented as part of schoolwide projects through activities such as needs assessment, staff 
development, changes in classroom instruction, and changes in school management (Millsap, et 
al., 1992). The presence of characteristics associated with effective schools may, reciprocally, 
impact the successful implementation of schoolwide projects, in that the factors that make good 
schools may also facilitate innovation and change. Case studies suggest that factors which 
facilitate innovation include: strong principal leadership and management skills, meaningful, 
universally agreed upon goals, a nurturing school culture, well-qualified staff, and organizational 
mechanisms to support schools problem-solving (Stringfield, et al, 1994). 

These preliminary findings lend some insight into the components which have been included as 
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part of schoolwide projects and also begin to form the basis for an understanding of what 
comprises a “successful implementation” of the schoolwide project approach. For example, one 
assessment describes successful implementation as schools which have “implemented schoolwide 
decision making, created effective working plans for improvement, integrated other existing 
categorical programs into a coherent instructional program for all children, allocated available 
resources more effectively, provided ongoing support to classroom teachers, learned how to 
monitor program effectiveness better, and instituted instructional reforms for their disadvantaged 
student population” (U.S. Department of Education, 1993b, p. 111). The larger, remaining 
questions, however, are what impacts schoolwide projects have had relative to traditional Title I 
service delivery approaches and their effectiveness in terms of improving student performance. 
Schoolwide Project Effectiveness and Student Performance 
Perceptions of Schoolwide Project Effectiveness 

Preliminary indicators of program effectiveness are reflected in school and district staff 
perceptions of the advantages and disadvantages of schoolwide projects. Concerns have been 
raised, for example, that school principals and district coordinators consider the primary 
advantages of schoolwide projects to be factors related to administrative convenience (e.g., 
reductions in class size, better use of materials and equipment), rather than to initiate instructional 
reform or improve student outcomes (U.S. Department of Education, 1992). Nonetheless, many 
principals and teachers consider the schoolwide project option as a way to do away with what they 
perceive to have been negative features of their previous Title I services such as pullout programs, 
formal or inflexible procedures for targeting Title I services, and an emphasis on remediation rather 
than prevention (Millsap, et al, 1993). Further, principals and district coordinators report a range 
of advantages, related primarily to instructional benefits for students and a simplification of 
administrative responsibilities through increased flexibility of programs and funds (Schenck & 
Beckstrom, 1993). 

The disadvantages reported most frequently by principals and district coordinators are: 
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additional paperwork; additional time for planning, evaluation, and teacher preparation; and the 
belief that Title I children receive fewer services (Schenck & Beckstrom, 1993). State and district 
Title I coordinators also have expressed the concerns that schoolwide projects may dilute the 
resources for Title I programs and that they do not adequately meet the needs of the lowest 
achieving students (Millsap, et al, 1992). 

Student Performance 

Information about the impact of schoolwide projects on student achievement is very limited. 

The only such information available at the national level is based on generalized principal and Title 
I district coordinator reports of schoolwide project effectiveness and on aggregate information on 
the number of schools which have met their three year accountability requirement. Seventy-eight 
percent of principals of schoolwide project schools which had been in operation for at least three 
years during the 1991-92 school year reported that evidence favored the schoolwide project 
(Schenck & Beckstrom, 1993). Of those schools considered to comprise the first group of 
schoolwide projects after the enactment of the Hawkins-Stafford amendments and assessed at the 
end of three years, only 10 percent failed to show the achievement gains required to continue as a 
schoolwide project (Turnbull, et al, 1992; U.S. Department of Education, 1993b). Finally, of a 
total of 128 schools in the Philadelphia school district, 40% of schoolwide project sites and 55% of 
non-schoolwide project sites were required to participate in mandated program improvement 
(Davidoff & Pierson, 1991). 

Richer information about student performance, however, is available at the local level, where a 
handful of studies of schoolwide projects have been conducted within particular districts (see 
Appendix A for more detailed summaries of these district-level studies). Studies have been 
conducted using data from the school districts of Philadelphia, Houston, Minneapolis, and a fourth 
unspecified urban school district which provide further insight into the impact of schoolwide 
projects on student achievement. These district-level studies focus primarily on comparison of 
NCE reading and math scores across schoolwide project and non-schoolwide project schools. Of 

14 




17 



those studies that conduct tests of statistical significance, most report only a handful of significant 
differences in measures of student performance between schoolwide and non-schoolwide project 
schools. Further, where significant differences appear, they tend to be small (see Table 3 for a 
summary of findings from four of these district-level studies). 

A study conducted by the Houston Independent school district, for example, compared reading 
and math achievement scores for its first year of schoolwide project implementation (Nechworth, et 
al, 1990). A handful of significant differences were identified, indicating higher reading and math 
scores in non-schoolwide project schools. The differences are slight, however. Another study in 
an unidentified major urban school district presents a mixed picture of schoolwide project 
effectiveness (Winfield & Hawkins, 1993). Following a cohort of first grade students for five 
years, the study found mostly non-significant differences in reading and math scores of students 
from the two types of schools. Significant differences were found among reading scores of 
students for two of the five years, suggesting mixed (both positive and negative) effects of 
schoolwide project status on reading achievement scores. 

A study conducted by the School District of Philadelphia, on the other hand, presents evidence 
in favor of schoolwide projects (Davidoff & Pierson, 1991). Although no tests of statistical 
significance are presented, a variety of comparisons favor schoolwide projects. Comparing four 
measures of student achievement during the 1988-89 school year, the study found that a higher 
percentage of schoolwide projects met the state-required measures of reading and math 
achievement than schools with traditional Title I programming. Further, following gain scores in 
reading and math across eight grades between the 1988-89 and 1989-90 school years, the study 
found that, for a majority of grades, a higher percentage of students from schoolwide project 
schools have higher post-test scores in reading and math than their counterparts in non-schoolwide 
project schools. 

The Philadelphia study also compared student performance in schoolwide project schools 
before and after schoolwide project implementation, finding improved performance subsequent to 
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implementation. The study showed that, for the majority of schoolwide project schools, the 
percentage of students receiving report card marks of “A”, “B”, or “C” across four curricular areas 
increased after schoolwide project implementation. Comparing average achievement gains in 
reading and math for the period prior to and after the implementation schoolwide projects, the 
study found that about half of schoolwide project schools had higher average gains in reading 
scores after implementation and about one-fourth had higher average gains in math scores. Lastly, 
three-fourths of schoolwide project schools showed improvement in student attendance after 
becoming schoolwide projects. 

Finally, a study of student achievement in schoolwide and non-schoolwide project schools in 
Houston (for the 1993-94 through 1994-95 school years) and Minneapolis (for the 1989-90 
through 1992-93 school years) found mostly non-significant differences in average NCE gain 
scores in reading and math for students from schoolwide and non-schoolwide projects (Wong, et 
al, 1996). Significant differences were found for average student math scores in the Houston 
school district which favor non-schoolwide project schools. However, comparison of math and 
reading scores over time for students in the Minneapolis school district suggests that students in 
schoolwide project schools were making gains at a faster rate than students in schools with 

traditional Title I programming. Another analysis from this study indicates that, for students in the 

\ 

Minneapolis School District, schoolwide project status contributed to equalizing differences in 
math achievement score gains between Title I eligible and ineligible students, when controlling for 
racial and social backgrounds. In the Houston School District, however, Tide I eligible students in 
schoolwide projects did not gain as much as they did in non-schoolwide project schools. 

Interpretative Cautions Regarding the Evaluation of Schoolwide Projects 

A number of factors should be taken into account before making conclusions based upon these 
or other evaluation findings. 

Lessons from the Initial Phase of Implementation 
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The Hawkins-Stafford amendments were passed in 1988 and implementation of the 
schoolwide project option did not begin in earnest in a number of sites until 1989. Thus, these 
findings are based, in most cases, on the first experiences of schools implementing schoolwide 
projects, and because broad-based school change is a slow process, they should be regarded as 
preliminary. The Houston Independent School District study, (Nechworth, et al, 1990) for 
example, presents test score data from the first year of schoolwide project implementation and is 
based only on this single year. Indeed, there is evidence that the presence of components of 
schoolwide project implementation is related to the number of years the schoolwide project has 
operated. For example, a study of an unidentified major urban school district found that hours of 
staff development activities were directly related to the year in which the schoolwide project started 
(Winfield & Hawkins, 1993). This study also found that the percentage of schools which 
reallocated resources to provide instruction to all students was increasingly higher over a three year 
period. As components of schoolwide projects become more integral to the life of schools beyond 
the initial phase, we expect evaluation results to be more meaningful. 

Methodological Issues Related to Schoolwide Project Evaluation 

The nature of Title I schoolwide project implementation presents a number of challenges to 
evaluation. First, students in schools with schoolwide projects are by definition, the neediest in 
their districts in terms of poverty, which is often linked with educational disadvantage. Therefore, 
finding an adequate comparison group of students from schools without schoolwide projects is 
difficult and students from comparison schools may demonstrate achievement differences which 
are linked to other factors. Further, high rates of student mobility in Title I schools create 
complexities for comparisons across schools as well as for longitudinal studies of particular 
cohorts of students. 

Also, as discussed earlier, these evaluations vary greatly in terms of their samples and 
consequent generalizability. Thus, the representativeness of study samples must be considered in 
the interpretation of any findings. The most generalizable findings are, of course, those based on 
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nationally representative samples. However, data from these samples tends to be limited, 
particularly with respect to measures of student performance. Great variability in instructional 
strategies within and across schoolwide approaches has also been documented which suggests the 
need to collect detailed information which is not often feasibly collected on a large scale 
(Stringfield, et al, 1994). Therefore, although small or non-random samples may be particularly 
useful for understanding the components of schoolwide projects which affect student learning, 
their limited generalizability must be recognized. 

School districts have a good deal of flexibility in the methods they use to meet schoolwide 
project accountability requirements and, indeed, the evaluations reflect a variety of methods. An 
analysis of data from students in the School District of Philadelphia highlights this flexibility, 
delineating and exploring the multiple ways a Philadelphia school can meet its accountability 
requirement (School District of Philadelphia, 1992). Although, most measures yield consistent 
decisions about continuation of schoolwide projects at particular schools, there are a number of 
inconsistencies across measures. Thus, it should be noted that some methods are better suited to 
detect schoolwide project effects than others. Similarly, data analysis methods also vary. Some 
involve traditional methods for establishing statistical significance, while others rely on more 
qualitative assessments of data. These differences should also be taken into account. 

Implications for Future Evaluation 

A number of implications follow naturally from these cautions. Evaluations need to continue 
beyond the initial phase of schoolwide project implementation and should be longitudinal in order 
to capture effects that may not be fully apparent during the first years. Careful selection of 
comparison groups and thoughtful sampling procedures will also increase the utility of subsequent 
evaluations. The variety of evaluation methodologies used thus far, however, is not necessarily a 
liability in that it provides a multifaceted picture of program effectiveness, using a variety of 
methods. Methodological studies which review the various methods used to measure student 
performance and meet accountability requirements for schoolwide projects have concluded that 
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multiple measures should be used (U.S. Department of Education, 1993c; School District of 
Philadelphia, 1992). Further, as student achievement expectations continue to broaden —departing 
from an emphasis on basic skills— current approaches to student testing may become increasingly 
limiting. 

A handful of evaluation studies devote attention to the role of the school district in the 
implementation of schoolwide projects. Case studies of schoolwide project schools indicate that 
school and classroom practices are shaped by district-level activities, such as encouragement and 
support for particular programs or for school-based management (Wong, et al, 1996; Stringfield, 
et al, 1994). Further, a national survey of school principal and Title I district coordinators 
indicates that the role of districts in the implementation of schoolwide projects varies greatly 
(Schenck & Beckstrom, 1993). Because schoolwide projects endeavor to support school-level, 
shared decision making, however, the appropriate role of the district is unclear. Evaluations 
should continue to explore the role of school districts and their impact on schoolwide project 
effectiveness. 

Because there is great flexibility afforded to schools with schoolwide projects, more attention 
should be given to the process through which changes at the school-level —resulting from 
schoolwide project implementation— impact student performance. Case studies of special 
instructional strategies implemented on a schoolwide basis indicate great variability in project 
implementation both within programs and across programs (Stringfield, et al, 1992). Therefore, 
studies which aggregate findings across school sites may indicate relatively little about the specific 
components related to their effectiveness. Evaluations need to pursue a better understanding of 
particular characteristics of schoolwide projects and the mechanisms through which they lead to 
changes in educational outcomes. 



Implications for Educational Administrators 

Schoolwide projects have the potential to address three interrelated challenges in the nation’s 



most disadvantaged schools. First, schoolwide projects provide increased flexibility to school-site 
professionals to address the "concentration effects" of disadvantaged pupils in poor 
neighborhoods. Second, schoolwide projects can reduce curricular and instructional fragmentation 
in the classroom. Third, and of immediate interest to national policymakers, schoolwide projects 
are designed to improve accountability at a time when there is growing public concern over the 
general quality of public education. 

Federal legislative expectations for Title I programs have been expanded to include challenging 
state content and performance standards (U.S. Department of Education, 1996). By the 1997-98 
school year, states will be expected to establish content and performance standards for Title I 
students which reflect the same knowledge, skill, and performance levels that are expected of all 
children. Title I program services must be aligned with the state content standards and, ultimately, 
states will be expected to develop or adopt student assessments which are aligned with their content 
and performance standards. 

Because Title I schoolwide projects incorporate a stronger accountability component, they offer 
the organizational potential to meet these new federal legislative expectations. Further, schoolwide 
project schools and districts should take the opportunity to think beyond basic accountability 
requirements and consider broadening the ways in which evaluation and assessment are used. For 
example, student assessment for Title I accountability purposes might also be used to guide 
instruction and improve teaching practice. Increased and alternative uses of student assessments 
might lead to more coordination at the district level and across units and departments. 

Dissemination of evaluation and assessment information as well as greater alignment of content and 
performance standards and assessments can help foster coherent links between curriculum, 
instruction, and assessment. 

Schoolwide projects also create a context in which roles of principals and district staff might be 
expanded or redefined. As discussed, schoolwide project schools have spent relatively short 
periods of time planning prior to implementation and have received mixed levels of support from 
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the district level. Further, although there is evidence that schoolwide projects have increased the 
number of hours of staff development, the content of the activities and their relationship to the 
implementation of schoolwide projects has not been emphasized. District staff might emphasize 
methods for phasing out pullout programs or for integrating traditional Title I reading and math 
curriculum with that of the whole school. 

The opportunity to redefine decision making roles at the school may also facilitate the creation 
of new structures which better serve students. For example, professional networks among 
teachers in the school might be fostered which encourage teachers to “buy into” aspects of the 
schoolwide project approach and cultivate changes at the classroom level. Similarly, schoolwide 
projects offer opportunities to explore broader governance issues. For example, alternative 
approaches to the functions of and relationships between the district, school, and classroom can be 
explored as well as particular types of parental involvement. 

The research on the effectiveness of Title I schoolwide projects in terms of student performance 
has yielded mixed and inconclusive results. Nonetheless, the fact that perceptions by district and 
school staff and decisions about continuation of schoolwide projects have tended to be favorable 
suggests that subsequent evaluations of schoolwide projects may begin to show more positive 
effects. It is also clear that the adoption of Title I schoolwide projects by schools and districts has 
been much faster than the availability of evaluation findings about their effectiveness. Despite the 
fact that there were over 8,000 schoolwide projects operating during the 1995-96 year, only a 
small handful of sites have produced reliable evaluation data. Thus, although the data presented 
here do not offer compelling evidence for or against schoolwide projects, it should be noted that 
these reflect only a small subset of the schoolwide projects operating in the nation. It is therefore 
critical that reliable, longitudinal evaluations continue to be conducted beyond this initial phase. 
Further, as state content and performance standards begin to be implemented more widely, they 
will offer continued opportunities to understand whether and how schoolwide projects impact 
student achievement. 
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Appended 

District-level Studies of Program Effectiveness and Student Performance 

Houston Independent School District (Nech worth, et al. 1990) 

The Houston Independent School District conducted an analysis of the effects of schoolwide 
projects on reading and math achievement scores using data from the 1989-90 school year. Data 
from a total of 42 schoolwide projects and 68 non-schoolwide projects are analyzed. Scores from 
students in grades 2 through 5 in both schoolwide and non-schoolwide projects schools are 
compared. Few significant differences were found, and where they exist, they are small. For 
non-limited English proficiency (LEP) students, slight significant differences were found which 
favor the non-schoolwide projects. In grade 2, both reading and math scores are significantly 
higher for students in non-schoolwide project schools. In grade 4, one of the two math scores is 
significantly higher for students in non-schoolwide project schools. And, when data from grades 
2 through 5 are pooled the math scores for students in non-schoolwide project schools are 
significantly higher. Using a different assessment tool for LEP students, the study found no 
significant differences in reading scores between students in the two types of schools and found 
only slight significant differences in math scores which, again, favor non-schoolwide project 
schools. 

Unidentified Major Urban School District fWinfield & Hawkins. 1993~) 

Another analysis, conducted of student data records from 40 schoolwide project schools and 
20 matched non-schoolwide project schools in a major urban school district, yielded mixed effects. 
Data were analyzed for students who entered the first grade in the 1986-87 school year, through 
their fifth grade year (1990-91). A regression analysis was conducted for each grade level using a 
school’s status as either a schoolwide or a non-schoolwide project school as a predictor of student 
achievement scores in reading. The analysis showed a mixed effect of schoolwide project status, 
finding significant differences in reading scores only for 2 grade levels — positive and negative 
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effects on reading achievement scores for grades 2 and 3, respectively. 

A second analysis, featured in the same report, was conducted using student data records for 
the school years 1986-87 through 1990-91 and survey data (collected during the 1989-90 school 
year) from 40 schoolwide projects in a major urban school district. This study linked student 
characteristics and components of schoolwide projects to student reading achievement scores. 
Three general categories of schoolwide project components were identified: 1) “school district 
framework” (e.g., inclusion of a school community coordinator), 2) “resource allocation” (e.g., 
providing classroom assistants), and 3) “existing programs” (e.g., computer lab). The total 
number of components within each category was added to create a scale indicating the presence of 
these components in sample schools. Several other components of schoolwide projects were 
measured using scales constructed from items on questionnaire mailed to principals including: 
degree of teacher involvement in decisions about how human resources are used; degree of teacher 
involvement in selecting basic material and purchasing instructional tools; degree of parent 
involvement in various types of activities; and percent of parents who attend parent-teacher 
conferences. 

The variables were entered into a regression equation using average student reading 
achievement scores for grades 1 through 5 as a dependent variable. The analysis adjusts for length 
of schoolwide project implementation and for other contextual and demographic characteristics of 
students and schools. Although no effect sizes are large, there are a number of significant effects. 
The components which appear to have the strongest effect are those included in the “district 
framework” category— having significant positive effects in all five grades except grade three, in 
which the effect is significantly negative. The “resource allocation” and “existing programs” 
categories of components each also show significant effects in at least two of the five grades— 
effects are positive for “existing programs” and mixed for “resource allocation”. The effects of 
teacher involvement in decisions about instructional resources, parent involvement, and parent 
conference attendance on student reading achievement are also significant (indicating both negative 
and positive effects) for at least two of the five grades. Teacher involvement in selection of 
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materials has a significant positive effect in only one of the five grades. 



School District of Philadelphia IDavidoff & Pierson. 19911 

The School District of Philadelphia conducted a number of studies which address the question 
of schoolwide project effectiveness. One study uses a data from a sample of 62 schoolwide project 
schools and 66 non-schoolwide project schools for the school years 1985-86 through 1990-91. 

The study considers the impact of the schoolwide projects on: student achievement in reading and 
math, report card marks, and average daily attendance. 

These analyses indicate that, for both reading and math, most evidence favored the schoolwide 
project sites over other Title I schools. Four measures of student achievement were considered, 
based upon statutory goals (required of Title I schools by the state of Pennsylvania ) during the 
1988-89 school year. Comparing the four measures for both schoolwide and non-schoolwide 
project sites, the study found that a higher percentage of schoolwide project sites met the four state- 
established criteria for reading achievement than did non-schoolwide project sites . In 
mathematics, however, a higher percentage of schoolwide project sites met only three out of the 
four criteria. 

In another comparison, the percentage of students with higher post-test scores in reading and 
math (administered in Spring, 1989 and Spring, 1990) and the average NCE gain scores for each 
grade from 1 through 8 are considered. These two indicators are compared for schoolwide projects 
versus all other Title I reading services. The comparison of the percentages of students with higher 
post-test scores in reading favors schoolwide projects in six of the eight grades. The comparison 
of average NCE gain scores in reading favors schoolwide projects in five of the eight grades. The 
results are similar for math, favoring schoolwide projects in six of the grades and seven of the 
grades for percent higher post-test scores and average NCE gain scores, respectively. 

Other analyses which compare student performance in schoolwide project schools before and 
after schoolwide project implementation also support schoolwide project effectiveness. Another 
analysis considered report card marks of students in schoolwide project schools, finding that in the 
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majority of the 37 “cohort 1 " schoolwide project schools in each curricular area (reading, math, 
science, and social studies), the percentage of students receiving a grade of “A”, “B”, or “C” 
increased after schoolwide project implementation. Comparing achievement gains in reading and 
math for the period prior to and after the implementation schoolwide projects, however, yielded 
less compelling findings about their effectiveness. In reading, only 21 of the 37 “cohort 1" 
schools (grades 2 through 8), demonstrated achievement gains as a schoolwide project which 
exceeded those during the three, prior pre-program years. In math, only 9 of the 37 schools 
demonstrated such achievement gains. Finally, 28 of the 37 “cohort 1 " schools had improved their 
attendance since becoming schoolwide projects. 

Houston and Minneapolis School Districts (Wong, et al. 19961 

Finally, another analysis compares the achievement scores of students in the school districts of 
Houston and Minneapolis. Data representing students from 35 schools (4 with schoolwide 
projects) in Minneapolis for the school years 1989-90 through 1992-93 were analyzed. A 
subsample was created that consisted of students who started the first grade in 1989 and remained 
in the same school until the third grade. Students were classified into six groups according to type 
of school (schoolwide project, advantaged Title I, or disadvantaged Title I) and student eligibility 
(Title I eligible or Title I ineligible). Average NCE gain scores for reading and math were 
compared across the six student groups for the years 1990-93. An analysis of variance (ANOVA) 
found no significant differences between mean gain scores for students attending different types of 
schools. Analysis of NCE test scores over a period of three years, however, suggests a narrowing 
of the gap between the performance of students in schoolwide projects and students in other types 
of schools over time. Although not a statistically significant finding, this suggests that students in 
schoolwide project schools were making gains at a faster rate than students in other types of 
schools. 

Data from 164 schools (116 with school wide projects) in Houston for the school years 1993- 
94 and 1994-95 were also analyzed. A subsample was created that consisted of students who 
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started the third grade in 1993 and remained in the same school until the fourth grade (1993 
cohort). Again, students were classified into six groups according to type of school (schoolwide 
project, advantaged Title I, or disadvantaged Title I) and student eligibility (Title I eligible or Title I 
ineligible). Average NCE gain scores for reading and math were compared across the six student 
groups for the years 1993-94. An ANOVA found no significant differences in the rate of academic 
growth among the six groups of students in reading; however, some differences were apparent in 
math. The largest gains in academic growth as measured by math achievement scores were made 
by Title I eligible and non-eligible students attending non-schoolwide project schools. An analysis 
of NCE test scores across the two years, identified a substantial (though non-significant) gap 
between the performance of Title I eligible and ineligible students. Comparison across school 
types shows that, regardless of school type, this gap was maintained across the two years. 

A hierarchical linear model (HLM) was also used to analyze NCE test scores for the 
Minneapolis and Houston samples. At the student level, NCE gain scores were modeled as a 
function of: starting scores, Title I eligibility, race/ethnicity, and poverty. At the school level, the 
average gain scores, effect of starting scores, Title I gap, minority gap, and poverty gap were 
modeled as a function of schoolwide project status. For students in the Minneapolis School 
District, when the effect of school type on academic growth is considered, no significant 
differences in average gain scores between schoolwide projects and non-schoolwide projects were 
found. When controlling for racial and social backgrounds, however, schoolwide project status 
contributed to equalizing differences in academic gains between Title I eligible and ineligible 
students, particularly in math. In other words, the gap between Title I eligible students and 
ineligible students was not as large in schoolwide projects, particularly in math. 

In the Houston School District, however, Title I eligible students in schoolwide projects did 
not gain as much as they did in non-schoolwide project schools. Particularly in math, the 
performance gap between students classified as Title I eligible and those classified as Title I 
ineligible within a school is larger in schoolwide projects than in non-schoolwide project schools. 
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The Effectiveness of Title I Schoolwide Projects: A Synthesis of Findings from the First Years of Evaluation 
Kenneth K. Wong and Stephen J. Meyer; ASCD, 1997 

Table 3: Summary of Findings from Four District-Level Studies of Student Performance Note: only significant findings or findings emphasized by report authors are presented. 
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